Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

schedule: avoid data race and panic about scheduler controller #7107

Closed
wants to merge 4 commits into from
Closed

schedule: avoid data race and panic about scheduler controller #7107

wants to merge 4 commits into from

Conversation

lhy1024
Copy link
Contributor

@lhy1024 lhy1024 commented Sep 18, 2023 •
edited
Loading

What problem does this PR solve?

Issue Number: Close #7106

What is changed and how does it work?

add a flexible wait group, because WaitGroup must ensure previous Wait has returned before added again.

Check List

Tests

  • Unit test

Release note

None.

@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Sep 18, 2023

[REVIEW NOTIFICATION]

This pull request has not been approved.

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot bot added do-not-merge/needs-triage-completed release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Sep 18, 2023
@lhy1024
Copy link
Contributor Author

lhy1024 commented Sep 18, 2023

/check-issue-triage-complete

@codecov
Copy link

codecov bot commented Sep 18, 2023 •
edited
Loading

Codecov Report

Merging #7107 (55adf55) into master (eb6953f) will increase coverage by 0.12%.
Report is 1 commits behind head on master.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #7107      +/-   ##
==========================================
+ Coverage   74.26%   74.38%   +0.12%     
==========================================
  Files         441      442       +1     
  Lines       47216    47234      +18     
==========================================
+ Hits        35064    35135      +71     
+ Misses       9063     9003      -60     
- Partials     3089     3096       +7
Flag Coverage Δ
unittests 74.38% <100.00%> (+0.12%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

@ti-chi-bot ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Sep 20, 2023
@lhy1024 lhy1024 changed the title schedule: avoid data race about scheduler controller schedule: avoid data race and panic about scheduler controller Sep 20, 2023
@lhy1024
add a flexible wg
0367c5b 
Signed-off-by: lhy1024 <admin@liudos.us>
@lhy1024
add some comments and tests
e3e346a 
Signed-off-by: lhy1024 <admin@liudos.us>
@HuSharp
Copy link
Member

HuSharp commented Sep 21, 2023

seems like DATA RACE in ci
https://github.com/tikv/pd/actions/runs/6252103050/job/16974605966?pr=7107

==================
WARNING: DATA RACE
Write at 0x00c0000b0098 by goroutine 37:
  github.com/tikv/pd/pkg/utils/syncutil.(*FlexibleWaitGroup).Add()
      /home/runner/work/pd/pd/pkg/utils/syncutil/flexible_wait_group.go:43 +0x107
  github.com/tikv/pd/pkg/utils/syncutil.TestNegativeDelta.func2()
      /home/runner/work/pd/pd/pkg/utils/syncutil/flexible_wait_group_test.go:82 +0x[35](https://github.com/tikv/pd/actions/runs/6252103050/job/16974605966?pr=7107#step:5:36)

Previous read at 0x00c0000b0098 by goroutine 35:
  github.com/tikv/pd/pkg/utils/syncutil.TestNegativeDelta()
      /home/runner/work/pd/pd/pkg/utils/syncutil/flexible_wait_group_test.go:86 +0x224
  testing.tRunner()
      /opt/hostedtoolcache/go/1.21.1/x64/src/testing/testing.go:1595 +0x238
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.21.1/x64/src/testing/testing.go:1648 +0x44

Goroutine [37](https://github.com/tikv/pd/actions/runs/6252103050/job/16974605966?pr=7107#step:5:38) (running) created at:
  github.com/tikv/pd/pkg/utils/syncutil.TestNegativeDelta()
      /home/runner/work/pd/pd/pkg/utils/syncutil/flexible_wait_group_test.go:81 +0x205
  testing.tRunner()
      /opt/hostedtoolcache/go/1.21.1/x64/src/testing/testing.go:1595 +0x2[38](https://github.com/tikv/pd/actions/runs/6252103050/job/16974605966?pr=7107#step:5:39)
  testing.(*T).Run.func1()
      /opt/hostedtoolcache/go/1.21.1/x64/src/testing/testing.go:1648 +0x[44](https://github.com/tikv/pd/actions/runs/6252103050/job/16974605966?pr=7107#step:5:45)```

nolouch
nolouch reviewed Sep 22, 2023
View reviewed changes
pkg/utils/syncutil/flexible_wait_group.go

// Wait blocks until the FlexibleWaitGroup counter is zero or negative.
func (fwg *FlexibleWaitGroup) Wait() {
fwg.Lock()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will it always hold the lock?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, fwg.cond.Wait() will work

@lhy1024
fix test
8ba64b8 
Signed-off-by: lhy1024 <admin@liudos.us>
@lhy1024 lhy1024 closed this Sep 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HuSharp HuSharp HuSharp left review comments

@rleungx rleungx rleungx left review comments

@nolouch nolouch nolouch left review comments

@JmPotato JmPotato Awaiting requested review from JmPotato

Assignees
No one assigned
Labels
release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

mcs: data race about scheduler controller
4 participants
@lhy1024 @HuSharp @nolouch @rleungx